Geocoder service returns latlong code for a given street address. geopy https://geopy.readthedocs.org/en/1.10.0 provides a number of different geocode service providers which we could use.
For Malaysia, Google's service returns the most accurate results, based on street names.
In [23]:
from geopy.geocoders import GoogleV3
In [24]:
street_address = "Jalan SS15/4G, Subang Jaya, Selangor"
In [25]:
api_key = "AIzaSyC0RKiMRtLlMWvFqQAbm5Z_qi43QwzrDdU" ##Get key for your own use from https://console.developers.google.com
geocoder = GoogleV3(api_key=api_key)
geocode = geocoder.geocode(street_address)
In [26]:
geocode
Out[26]:
In [27]:
reversed_geocode = geocode[1][::-1]
reversed_geocode #MapIT API uses this form
Out[27]:
We now have a geocode in format that Mapit service can use to give us information on electoral boundaries.
MapIT service a Poplus component returns administrative boundaries via different API calls, one of which is geocode location.
http://mapit.sinarproject.org/#api-by_point
By providing this geocode to MapIT, it will provide us with a list of boundaries this geocode location point is in.
REST API works almost the same as your brower requests, in fact you can test it out in your browser to get results.
JSON is standard format for getting data that is easy to work with, and most open data APIs return data in this format.
In [28]:
import requests #easier to use requests library than urllib2 to make http requests
In [29]:
import json
#We're making API request to get a list of all parliamentary areas in Malaysia
PARs = requests.get('http://mapit.sinarproject.org/areas/PAR')
PARs.content
PARS_json = json.loads(PARs.content)
This get's us all the information on parliamentary areas stored in Sinar's Malaysian MapIT instance. The id keys is just to test that we have them all, and we will use this key to match later.
In [30]:
PARS_json.keys()
Out[30]:
In [31]:
location = '%s,%s' % (reversed_geocode[0],reversed_geocode[1])
areas = requests.get('http://mapit.sinarproject.org/point/4326/' + location)
areas_json = json.loads(areas.content)
We now have in areas_json a set of areas and key area id's just like the list of parliamentary areas before. Except this result shows all sorts of areas, DUN, local council and what not.
http://mapit.sinarproject.org/point/4326/101.589965,3.073426.html
We have DUN: http://mapit.sinarproject.org/areas/DUN
Local Councils: http://mapit.sinarproject.org/areas/ZON
Daerah Mengundi: http://mapit.sinarproject.org/areas/DM
State: http://mapit.sinarproject.org/areas/STT
For Python, we use a set, to find the area id/info that is the same, which should be a unique PAR area. In other languages you can use other methods of matching ids.
In [32]:
PAR_set = set(areas_json).intersection(set(PARS_json))
PAR = PAR_set.pop()
In [33]:
constituency = PARS_json[PAR]
print constituency['name']
print constituency['codes']['code']
#can't get full name without area lookup due to https://github.com/mysociety/mapit/issues/189
In [ ]:
Now that we have the constituency code, we can lookup Sinar's Popit database for Posts such as MP matching this code.
In [43]:
#search for posts for constituency P104
posts = requests.get("http://sinar-malaysia.popit.mysociety.org/api/v0.1/search/posts?q=%22" + constituency['codes']['code'] + "%22")
posts_json = json.loads(posts.content)
In [105]:
import datetime
from dateutil import parser
start_date = "2013-05-16"
start_datetime = parser.parse(start_date)
members = posts_json['result'][0]['memberships']
def current_rep_id(members):
#returns id of current parliamentarian
for member in members:
elected = parser.parse(member['start_date'])
if elected >= start_datetime:
return member['person_id']
#now let's get our MP
mp_raw = requests.get("http://sinar-malaysia.popit.mysociety.org/api/v0.1/search/persons?q=id:" + current_rep_id(members))
mp_json = json.loads(mp_raw.content)
mp = mp_json['result'][0]
In [113]:
print mp['name']
print mp['contact_details'][0]['value']
print mp['image']
=Credits and Notes=
This is just a raw example in interactive iPython Notebook format http://ipython.org/notebook.html
It is encouraged that others write and share similar tutorials or helper libraries in different langauges such as PHP, Ruby, Go, Java and more based on this walkthrough.
If you do implement some lookup functions using Popit or Mapit services for Malaysia do contact us at team@sinarproject.org
Credit goes to Tindak Malaysia for generously donating their electoral boundary data to be made as open data for use of all in our MapIT service.
Other ideas:
Lots of data sets have street addresses such as construction http://sinarproject.org/en/projects/scrapers-1/construction-industry-development-board-cidb-database